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Abstract: Medical plants play a vital role in curing many diseases. These plants, along 
with their leaves, have medicinal values. If these leaves are identified appropriately, they 
can be chosen directly to have more significant relief from the disease. Therefore, the 
identification of these medical species is a challenging task. The ecologically motivated 
Convolutional Neural Networks (CNNs) have substantially contributed to computer visual 
research. This article introduces a unique approach to medical leaf identification based on 
the hybridization of the Gaussian mixture model and a Convolutional Neural Network 
(GMM-CNN). The experimentation is performed on the Flavia dataset and is carried out 
using benchmark evaluation metrics. The parameters like index volume, probability of 
random index, and global consistency error are evaluated. The Python simulation model is 
utilized for the evaluation of the proposed methodology. The hybrid technique combining 
GMM and CNN has considerable potential in medical leaf identification. The experimental 
findings indicate that the hybrid strategy exhibits superior performances. The methodology 
suggested in this study demonstrates exceptional levels of accuracy, precision, and recall 
when applied to a wide range of medical leaf categories. Moreover, integrating Gaussian 
Mixture Model (GMM) and Convolutional Neural Network (CNN) addresses concerns 
associated with a scarcity of training data by offering a more resilient structure for 
extracting features and performing classification. By combining the advantages of 
Statistical modelling with deep learning, we develop a resilient and precise system that has 
the potential to enhance botanical research, medical diagnostics, and environmental 
monitoring applications. 


and effective categorization are system capabilities 


The phrase "computer vision" refers to a broad 
concept that depicts a computer acting as an eye by 
applying various algorithms with mathematical model to 
a digital picture. In this field, photographs from the actual 
world are processed automatically while features are 
extracted and information is immediately interpreted by 
user needs (Sarraf et al., 2021). Image recognition is the 
core function of computer vision (Yang et al., 2021). The 
ability of human eyesight to quickly recognize and 
distinguish between the items around it makes it 
outstanding and unique. Perfect 3-D structure perception 


(Welchman et al., 2016). Despite being modelled after 
the human vision framework, computer vision (CV) is 
still more effective at detecting, identifying, and 
differentiating things. The CV can be seen as an 
approximate representation of human vision since it is 
such a sophisticated system (Rainey et al., 2021). 

The science of plant taxonomy focuses on acquiring, 
recognizing, explaining, 
plants. Techniques 


characterizing, 
used in this 


and naming 
field include 
morphological, anatomical, and chemotaxonomic 
classifications (Sing et al., 2018). Morphological and 
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anatomical categories are seen to be more conventional 
than chemotaxonomy (Simpson, 2019). The two main 
components that play essential roles in plant taxonomy 
are the categorization of plant and the authorization of 
plant, which deal with identifying a recently noticed plant 
concerning a formerly collected specimen and placing a 
known plant in a group depending on its similarities with 
other plants. 

Random noises will be injected into the obtained 
pictures as a result of the impacts of variable lighting and 
throughout picture capture, transmission, and 
digitalization, there are changes in sensor temperature, 
and these noises usually have Gaussian characteristics (Li 
et al., 2018; Feng et al., 2020). The presence of noise will 
not only impact the aesthetic appeal of the picture. 
However, it will also impact future image processing, 
including feature extraction, classification, and 
identification, among other things. Denoising must thus 
be applied to the captured picture before image 
processing to enhance the image's quality and make post- 
processing easier. 

Traditional medical practises have long made use of 
medicinal plants. In order to detect medicinal plants, 
many real-time vision systems have recently been created 
employing computational approaches and machine vision 
applications (Azadnia et al., 2021). Since deep learning 
(DL) approaches can manage feature extraction and 
picture selection concurrently, they have primarily been 
used. Because of this, DL has recently become quite 
popular in various farm automation applications where 
object recognition and picture categorization are 
necessary (Azadnia et al., 2022). One of the most 
effective DL techniques is the convolutional neural 
network (CNN), which has shown to be highly effective 
in pattern recognition and picture segmentation 
(Yamashita et al., 2018). The CNN employs properly 
trained layers for the identification of plants and 
categorization (Ghazi et al., 2017) and the detection of 
plant diseases (Arsenovic et al., 2019). Low-level 
features like colours, corners, and edges are learnt from 
the first layer of CNN's hierarchy of self-learning 
characteristics. In contrast, the deep layer teaches high- 
level properties like textures and objects of the image. 
The sensitivity of CNNs to environmental factors like 
variations in light has been decreased through automated 
feature learning. 

Medical leaf identification is of greater importance as 
these leaves help in the preparation of drugs, and some of 
these leaves can be taken in raw form to help cure 
diseases. However, there are issues concerning the 
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identification of these medical leaves. Some of the most 

significant issues include; 

e Identifying a leaf and ascertaining whether it belongs 
to the group of medical leaves. 

e Identification of the medical leaf despite certain 
constraints such as degradation, speckle noise, color 
changes, orientation changes, changes in shape, etc. 
To meet the above challenges in this proposed work, 

we present a methodology wherein the input leaf is pre- 

processed, then denoising using GMM model, and finally 

a neural network system is utilized to extract the features 

of the leaf. Various parameters are been evaluated to 

support and analyze the performance of proposed 
methodology. 


Related works 

The classification process is presented as a unique 
CNN architecture in Habibollah (2016). The author 
arbitrarily selected 20 leaves from 40 different species to 
represent the front and back of Ayurveda. Weka is a 
program used to recognize medicinal plants using 
machine learning methods. Based on these metrics, SVM 
and MLP classifiers are employed to identify the leaves. 
Using neural networks (Li et al., 2018), authors devised a 
method for identifying medicinal plants. As expected, 
MLP outperformed SVM (94.5%). Consideration is given 
to the leaves of five distinct plant species. To determine 
the location of the leafs edges, the Prewitt Edge 
Detection method is utilized. An overview of the 
techniques for classification used for recognizing 
medicinal plants using ANN, SVM, and PCA (Chau et 
al., 2017). In order to distinguish neem leaves by their 
shape, colour, and vein properties, the authors have 
assessed the angle percentage, centroid, region, edge, and 
roundness in their proposed study. Sabu et al. (2017) 
proposed a new classification scheme for Ayurvedic 
herbs based on the characteristics of their leaves’ 
morphology is proposed. The Laplacian filtering 
approach is used for edge detection. 

A deep CNN is suggested as a solution by (Grinblat et 
al., 2016) to correctly classify plants based on the vein 
patterns in their leaves. The precision of the suggested 
pipeline is significantly improved by utilizing this deep- 
learning method. In addition to this, they provide 
evidence that the accuracy that has been claimed is 
attainable by increasing the model depth. A deep learning 
algorithm for classifying leaves was suggested, replacing 
the red channel of colors with leaf pre-processing to 
obtain vein form information (Huynh et al., 2020). Both 
the Flavia leaf data set and the Swedish leaf data set, 
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which are collections of actual leaves, were used to verify 
the accuracy of this model. 


Input Data 
Collection 


Feature 
Extraction 


Feature 
Classification 


The author has developed an autonomous system for 
diagnosing diseases in potato plants utilizing CNN 


Image Pre- 
processing 


Gaussian 
Mixture Model 


Convolutional 
Neural Networks 


Parametric 
Evaluation 


Figure 1. Process flow of the proposed model 


The author (Wang et al., 2021) created AMSCNN, a 
novel multi-scale CNN, to enhance the detection of 
species of plants. To improve feature extraction and 
network training efficiency, this network is built to learn 
lower and higher-frequency features from the given 
images as input to the system via multi-scale convolution, 
and it incorporates an attention mechanism to capture 
complicated contextual interactions. To classify photos of 
medicinal plant leaves (Azadnia et al., 2022), describe a 
Deep Learning (DL) model that utilizes CNN block to 
extract features. The classifier block is comprised of 
layers of Global Average Pooling, thick layers, dropout 
layers, and softmax layers. Images of medicinal plant 
leaves recorded at varying resolutions were used to test 
the algorithm (64x64, 128x128, and 256x256 pixels). 

A disease detection model by (Shelar et al., 2022) uses 
image processing with a CNN to aid in detecting plant 
illnesses. The study is driven by the fact that pests can 
majorly affect agricultural output and that diagnosing 
plant illnesses is a laborious and inaccurate process. To 
overcome this difficulty, the researchers recommend 
utilizing a convolutional neural network (CNN) to handle 
pixel inputs like photos. A hybrid model is evaluated to 
detect plant disease (Bedi et al., 2021), which was trained 
and validated with the publicly accessible Plant village 
dataset. The suggested hybrid model's combination of 
deep and machine learning approaches; 
represents an original contribution to the field of plant 


learning 


disease identification. 
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architectures. The different types of networks that can be 
modelled for the extraction of features are VGG-19, 
VGG-16, and InceptionV3 and different classifiers are 
utilized for classification. Some of them are Logistic 
Regression (LR), k-Nearest Neighbor (KNN), Support 
Vector Machine (SVM), and Neural Network (Mohameth 
et al., 2020). The researchers determined that the 
combination of WVGG-19 and Logistic Regression 
produced the highest accuracy of 97.8% based on their 
findings. The author proposed an automated system for 
identifying medicinal plants in the Borneo region. It uses 
a computer vision system with a deep learning model 
trained on a dataset of plant images, a knowledge base, 
and a mobile application for user interaction (Tiwari et 
al., 2020). The author discussed how to use deep 
convolutional neural networks (CNN) to identify the 
leaves of medicinal plants (Malik et al., 2022). However, 
a deep CNN model can be developed and improved to 
speed up the identification process. Multi-layer models 
and data augmentation (including image cropping and 
rotation) dramatically increased precision in this research. 


Proposed methodology 

The approach contributed to this work mainly utilizes 
three core steps after collecting the data that need to be 
considered for evaluating the work. The first step is pre- 
processing, second is image denoising and_ finally 
features extraction and classification. The process flow of 
the work is shown in Figure 1. 
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Data collection 

The Department of Health's list of medicinal plants 
includes oregano, bayabas, yerba buena, ulasimang-bato, 
ampalaya, malunggay, sambong, lagundi, tsaang-gubat, 
and niyog-niyogan. This investigation focuses on the list 
of medical applications that have been granted. Images of 
the sample herbal plant used in the investigation are 
shown in Figure 2; a total collection of 600 images was 
gathered. The leaf dataset utilized is flavia. 
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Figure 2. Flavia Dataset 


data, and estimate the features from the provided dataset. 
GMM resources may be used to determine how many 
components should be present in each leaf. 


Negative log-likelihood predicted by a GMM 
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Figure 4. GMM model for extracting features 


A "Gaussian mixture model (GMM)" is a probabilistic 
framework that assumes each data point was generated by 
blending a small number of Gaussian distributions with 


Figure 3. Pre-Processing output sample 


Image preprocessing 

In this step, all the images in the dataset are 
considered and resized for performing further operations. 
The leaf images are pre-processed to remove unwanted 
information, such as the image's background, and extract 
the image's foreground. For this purpose, a particular 
threshold value is selected and applied on the leaves to 
separate the background. To identify the optimal 
threshold value OSTU model is utilized (Vala et al., 
2013). Several pixels are identified in the image in which 
some of the empty pixels in the leaf are suppressed using 
morphological dilation operation. The background noise 
present in the leaf will be removed using a smoothing 
filter. After this stage, the data is sent for feature 
extraction. 
Image feature extraction 

Gaussian Mixture Models (diagonal, tied, spherical, 
and full covariance matrices supported) sample the leaf 
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unknown features. As an extension of k-means clustering, 
mixture models are a way to account for the structure of 
covariance in data and the locations of latent Gaussian 
centers. 

The GMM is examined in this article as a means of 
accurately identifying the characteristics of medicinal 
plants. These features are essential to classify the plant 
labels. Every plant leaf attributes a symmetric pattern; 
hence, to model these sorts of plant leaves, a bell-shaped 
or symmetric distribution is well suited. Therefore, in this 
article, we have considered the GMM model. 

The probability distribution function of GMM is given 
by 
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where pt and o are the parameters inside the image 
regions. To have a better approximation of the features, 
the initial estimates obtained by the GMM are to be 


GMM Features 


Leaf image Generation 


Machine Learning algorithms. The next step is to classify 


the data using CNN. 


GMM feature 
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features 


Figure 5. Selection of targeted features from the dataset 


processed and updated. The GMM utilizes the EM 
technique for fitting a Gaussian mixture model. It can 


evaluate the availability of original data and produce 
confidence ellipses for multivariate models. 


Features classification 

The process of feature classification is performed 
using CNN. The final layer present in the CNN acts as 
classifier. The features need to be extracted for the given 
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Figure 6. CNN model for suggested work (R. Azadnia et al., 2022) 


Every plant leaf has different shape and colour 
features. To have more approximation of identification 
among the medical leaves from the dataset, the leaves 
have different orientations, degradation, damages, etc. 
The leaf should be identified more concurrently. For this 
purpose, effective modelling is necessary. Therefore, the 
Gaussian Mixture Models are considered to model the 
leaf images precisely in this article. Another advantage of 
the GMM is that it accurately estimates the leaves with 
symmetric properties. Therefore, with this approximation, 
the GMM is proposed. 

However, to better understand the pixels’ connectivity 
and the relationship between the pixels, the accuracy can 
be much more increased. The Convolutional Neural 
Networks algorithm has this advantage among the 
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input image to identify the type of medicinal plant. The 
classification of features helps in finding the accurate 
measure of parameters evaluated. The model of CNN 
considered for evaluating the proposed work is shown in 
Figure 6. 

CNNs are sophisticated deep-learning algorithms that 
excel at image analysis. They excel at object detection, 
image classification, and segmentation in computer 
vision. CNNs filter or kernel input data. Filters recognize 
edges, corners, and patterns. A feature map is created by 
convolving the filters with the input data. After each 
filter, an activation function like ReLU adds non-linearity 
to the network, letting it learn more complex patterns and 
features. The steps involved during the performance of 
the CNN process are stated below: 
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Step 1. The first layer of a convolutional neural 
network (CNN), the input layer, takes in the original 
picture data. Most of the time, the input data is shown as 
a 3D array of pixel values. 

Step 2. In the convolutional layer, filters are applied 
to the input image, each looking for a different feature 
type. A feature map is made by sliding the filter across 
the image and multiplying its values by the pixels’ values. 

Step 3. The pooling layer down-samples the feature 
map, decreasing its dimensionality in space. The fewer 
parameters a network has, the better it will perform 
computationally, and this helps. 

Step 4. The output of the convolutional layer is 
applied to a non-linear activation function in the 
activation layer. This gives the network some non- 
linearity, which improves its ability to learn complicated 
characteristics. 

Step 5. In order to make a fully connected layer, each 
neuron in the prior layer must have a connection to each 
neuron in the subsequent layer. Using the preceding 
layers' output as input, this layer learns more abstract 
characteristics. 

Step 6. The final output layer of the network predicts 
the class or label of the image sent in. 

A complete control across the entire dataset utilized 
for trainingis referred to as an epoch. The CNN 
iteratively adjusts its parameters during each epoch to 
minimize the loss function and improve its accuracy. By 
training the CNN with many epochs, it can learn to detect 
complex image features and make accurate predictions. 
To avoid underfitting or overfitting, it's crucial to find a 
balance between training for an appropriate number of 
epochs and not enough epochs. 

Image classification 
revolutionized computer-vision 


has 
enabling 


using CNN 

tasks by 
automated and accurate recognition of objects within 
images. CNN-based image classification algorithms have 
gained immense popularity due to their ability to 
automatically learn and extract intricate features from 
raw image data. Image classification involves assigning 
labels or classes to input images. It is a supervised 
learning task where a model is trained on labelled image 
data to predict the class of unseen images. CNNs are 
commonly used to classify images as they can learn 
hierarchical features like edges, textures, and shapes, 
enabling accurate object recognition in images. CNNs 
excel in this task because they can automatically extract 
meaningful spatial features from images. 
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Metrics evaluated 

Performance metrics like Average Distance (AD), 
Maximum Distance (MD), Image Fidelity (IF), and 
image segmentation metrics like volume of index (VOD, 
Probability Random Index (PRI), and Global Consistency 
Error (GCE) have been used to assess data precision. 
These equations define the evaluation of the metric. 

Average Distance (AD): 

AD=(1/N)*>\d 


Here, N represents the total number of pixels in the 
ground truth region, and d represents the closest point on 
the segmented border to the ground truth boundary. 

Maximum Distance (MD): 

MD = max(d) 

Where d is the same as in the AD formula, and max(d) 
is the maximum distance between the segmented 
boundary and the ground truth boundary over all pixels in 
the ground truth region. 

Image Fidelity (IF): 

IF =10* log 1O(@MAX%2)/ MSE) 

MSE is the mean squared error between the 
segmented image and the ground truth image, and MAX 
is the maximum pixel value (255 for an 8-bit image). 

Volume of Index (VOD: 

VOI(A, B) =[VOI(A) + VOI(B)]/2—VOI(A 0 B) 

Where A and B are the segmented regions, A N B is 
their intersection, and VOI(A) and VOI(B) are given by: 

VOI(A) =—log(V(A)) 
VOI(B) =—log(V(B)) 

Where V(A)andV(B) are the volumes of the 
segmented regions A and B, respectively. 

Probability of Random Index (PRI): 

PRI(A, B) =[PRI(A) + PRI(B)]|/2— PRI(A XB) 

Where A and B are the segmented regions, A N B is 
their intersection, and PRI(A) and PRI(B) are given by: 


PRI(A)=)[pG, J)* p@. 7) > pGA*> EF) 


PRI(B) => [pG )* PG. J) > PG D* > PG. 7) 
Where p(i,j) is the probability of a pixel in region 
A and p(i’,j’) is the probability of a pixel in region B. 
Global Consistency Error (GCE): 
GCE=1- [> cmin( d(A,i),d(B,i)))/ Yi amax(d(A, i), d(B,i)))] 


Where A and B are segmented regions and 
d(A,i) and d(B,i) are the distances of the ith pixel in 
the image to the boundary of regions A and B, 
respectively. The numerator adds the minimum distances 
between each pixel and its region's boundary, while the 
denominator adds the maximum distances. 
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Results and discussion 

The findings show that the suggested GMM-CNN 
algorithm can detect medicinal plants with comparable 
leaves successfully. The experimentation is considered in 
a Python environment, and around 350 medical images 
are considered for training the data. For testing purposes, 
around 200 medical images are considered. The results 
derived are presented in Table 1. 
Table 1. Evaluated parameters 


Without 
GMM 
Model 


Quality 


Proposed 


Name of Leaf Method 


Imz f leaf 
mage of lea Metric 


AD 0.678 0.802 
Tulasi Leaf MD 0.61 0.882 
IF 0.388 0.786 
AD 0.61 0.88 
Curry leaf MD 0.25 0.81 
IF 0.35 0.85 
AD 0.58 0.73 
Hibiscus 
leaf MD 0.62 0.85 
IF 0.56 0.88 
AD 0.35 0.68 
Thanduvalai MD 0.37 0.85 
oe iF | 033 | 0.86 
VOI | 2.453 | 5.4523 
Neem leaf | GCE | 0.234 | 0.4321 
PRI 0.654 0.674 
VOI | 0.267 0.768 
Tulasi leaf | GCE | 0.056 | 0.0834 
PRI 0.598 0.678 


A number of problems, such as a large cost of 
computation, difficulty, and a long duration of 
processing, have made it difficult to employ the primary 
CNN models. We employed two alternative strategies to 
address this problem: (1) increasing the amount of 
smaller-sized and more numerous pictures used during 
the operation of pre-processing of the image and (2) 
FC 
classification of features, drastically paring down the 
model's complexity and number of variables. Finally, it 


utilizing the layer for both extraction and 


has been demonstrated that these methods boost the 
model's precision and speed of computation while also 
reducing the risk of the over-fitting problem. The 
comparison of parameter values is shown in Figure 7. 
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comparison of parameters values 
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Tulasileaf curreyleaf Hybiscus leaf Thanduvalai 


leaf 


BAD a MD GIF 


Figure 7. Comparison of values obtained for different 
leaves 


Conclusion 

In botany and agriculture, it is crucial to differentiate 
between medicinal plants and other types of plants that 
humans do not consume. Identifying medicinal plants 
using conventional methods is laborious, time 
consuming, and requires experts. An autonomous, real- 
time, vision-based approach that could identify frequently 
used plants for medicinal purposes with similar leaves 
had favorable outcomes. The suggested solution uses a 
convolutional network, an upgraded CNN version. The 
integration of Gaussian Mixture Models (GMM) with 
Convolutional Neural Networks (CNN) effectively 
mitigates certain constraints inherent to each method 
when employed in isolation. The Gaussian Mixture 
Model (GMM) provides 
representing complicated data distributions, enabling the 
system to effectively capture nuanced changes in leaf 


a robust framework for 


characteristics. In contrast, Convolutional Neural 
Networks (CNNs) have exceptional proficiency in 
autonomously extracting hierarchical features from 
unprocessed picture data, hence facilitating the system's 
ability to acquire discriminative patterns straight from 
images. The hybrid model combines the utilisation of 
feature extraction from Convolutional Neural Networks 
(CNNs) with the probabilistic modelling capabilities of 
Gaussian Mixture Models (GMMs) in order to generate a 
more complete representation of leaf attributes. The 
model could recognize the medical species even in 
rotation, orientation, and shape patterns. For evaluation, 
we have considered different evaluation metrics like 
Average Difference, Image Fidelity, VOI, GCE, and PRI, 
and the results showcase better improvement using the 
proposed model. 


Nevertheless, it is crucial to recognise that there are 
still obstacles that need to be addressed. Ongoing 
research efforts are focused on the refinement of 
parameters for the hybrid model, optimisation of 
computing resources, and the establishment of scalability. 
Furthermore, it is necessary to do further research on the 
interpretability of the model's decision-making process 
and its potential applicability to different areas. 
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